Excellence in Evaluation: Early Landmarks at the National Library of Medicine

F. Wilfrid Lancaster has earned a reputation for greatness in the evaluation of information storage and retrieval systems. Many of his extensive contributions stem from his early experience with the National Library of Medicine (NLM) MEDLARS system. His evaluation of the MEDLARS Demand Search Service in 1966 and 1967 was an important landmark as one of the earliest evaluations of a computer-based retrieval system and as the first application of recall and precision measures in a large, operational database setting. In 1971, his evaluation of the MEDLARS AIM-TWX system was an important study of early online systems and their direct use by end users. This paper summarizes Lancaster’s two major evaluations of the MEDLARS system, including the information environment at the time and their impact in the field of information science. Examples of Lancaster’s other evaluation work with information retrieval systems are provided, followed by discussion of the textbooks that grew out of his evaluation experience and expertise. The article closes with comments from current and former NLM staff regarding Lancaster’s time at NLM or his influence on their own career. Introduction F. Wilfrid Lancaster established himself as a giant in the evaluation of information storage and retrieval systems early in his career, and his reputation for greatness in this arena stands today. Many of Lancaster’s extensive contributions stem from his experience with the National Library of Medicine (NLM) MEDLARS system. As one of the earliest evaluations of a computer-based retrieval system, his evaluLIBRARY TRENDS, Vol. 56, No. 4, Spring 2008 (“The Evaluation and Transformation of Information Systems: Essays Honoring the Legacy of F. W. Lancaster,” edited by Lorraine J. Haricombe and Keith Russell), pp. 859–887 (c) 2008 The Board of Trustees, University of Illinois Excellence in Evaluation: Early Landmarks at the National Library of Medicine